|
|
|
 |
|
|
How do I remove my page from HotBot?
HotBot honors the "robots.txt" file standard, which is documented at http://info.webcrawler.com/mak/projects/robots/norobots.html. This file can be placed on your site to tell search robots which directories they should add to their databases and which they shouldn't index. If you prefer that your site not be indexed by HotBot, ask your webmaster to create a robots.txt file for your site. HotBot's crawler will fetch and obey this command file. It will obey any entry with a user agent of "*" or containing the word "Slurp" (the name of HotBot's crawler). HotBot also honors the "robots noindex" meta tag, which keeps HTML files out of HotBot's database index. This can be added to the head section of an HTML document, as illustrated below. <HTML>
<HEAD>
<TITLE>A Document</TITLE>
<META NAME="ROBOTS" CONTENT="NOINDEX">
</HEAD>
<BODY>
[the text of your document]
</BODY>
You can also simply remove the offending page from your Web server or restrict public access to your server (talk to your server administrator, system administrator, or dealer if you don't know how to do this). Your page listing will remain on HotBot until the next time the Web crawler visits your site, which can take up to 60 days. We do not manually remove pages from our index for the security of people submitting pages to HotBot. If you find a page that you take issue with (your personal information displayed, trademark infringements, etc.), please write to the webmaster at the domain hosting the page rather than HotBot Support. Whereas the webmaster of a host domain can have the page taken down, HotBot Support cannot take action based on claims by a third party of legal infraction or other abuse.
Common Questions | Search Tips | HotBot Tools | Customer Support Add URL | Send us feedback |
Work at HotBot |
Advertise on HotBot |
Company Information |
Our Privacy Policy |
Help |
|